Some conventional image acquisition systems have the capacity to combine individual images for the purpose of producing composite images that detail an enlarged field of view. These image acquisition systems use methodologies that rely upon the capture of the images by one or more cameras. In order to combine the images that are captured, some conventional systems rely on the overlap of image regions of the captured source images.
The quality of a composite image is constrained by the imagery that is used in its creation. It should be appreciated that the resolution involved and the number of viewpoints that are considered are important factors that impact the creation of composite images. The greater the resolution and number of viewpoints provided the greater the spatial resolution of the resultant composite image. While digital still cameras are reaching mega-pixel dimensions at nominal cost (e.g., providing increasingly high resolution images), the spatial resolution provided by digital video systems lags far behind that offered by digital still cameras.
The creation of composite images (e.g., mosaicking) involves combining source images captured from a plurality of camera viewpoints. The source images are derived from viewpoint associated video streams and are used to form the composite image. A conventional approach to the creation of composite images involves finding points that correspond in the contributing images and computing stitching homographies that relate their perspectives. This approach derives from the situation where images are collected from arbitrary positions, such as in hand held capture. There, the features for deriving each homography must come from the acquired images themselves. If the camera views share a center of projection, the features can be chosen from anywhere in the overlapping images and their homographies will be valid throughout the scene viewed. However, when they don't share a projection center, the features must be collected from a shared observation plane and the homography may only produce seamless composite images for imagery in that plane.
For the reasons outlined above, conventional systems that composite images are relegated to low-resolution implementations that employ a limited number of viewpoints. The limited number of viewpoints provides a limited capacity to produce panoramas from acquired images that have high spatial resolution. The performance of conventional systems is further limited by their reliance on the use of overlapping image data to generate homographies. The requirement that the source images used to compose a composite image overlap decreases the size of the view angle that can be imaged as it prevents the imaging of non-overlapping views that can cover a wider measure of space.
One skilled in the art will appreciate that the quality of such a composite image will depend on various factors including the separation between the imagers, their focal lengths, and the distance of observed features from the shared observation plane to which the camera is “focused.” A scene whose content is aligned with such plane will be smoothly mosaicked while one whose content varies from that plane will have discontinuities at those deviations, and the magnitude of those deviations will increase with increasing distance from the plane.
One familiar with the fields of image processing and computer graphics will appreciate that most scenes are not aligned with specific planes and in fact have content at arbitrary locations. If that content is aligned with some other plane, then a smooth mosaic could be obtained through registration, as will be described, with respect to that plane. On the other hand, if the content has variable depth, then it may be that no single plane would suffice for its mosaicking and a piecewise subdivision of the scene into smaller planes where the content is adequately represented as planar may provide composite images that are smoothly mosaicked.